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(54) Abstract Title 

Three-dim ensional sound and graphics reproduction system 



(57) A host processor unit (101) of a personal computer or game machine transfers scene data, drawing 
commands and sound commands to a geometric computation unit (102). The host processor unit or the 
geometric computation unit generates two-dimensional coordinate and colour data therefrom for a display, 
and the geometric computation unit identifies by computation the geometric relationship between a sound 
source and a point of listening. A sound generator (105) reads out, e.g. from a sound memory (104), sound 
data of a sound source specified by a sound command and reproduces sound in a three-dimensional space by 
processing the sound date, based on the geometric information from the geometric computation unit relating 
to the sound source and the point of listening. The sound data may relate to ambient sound, diffuse sound and 
directional sound. 

HG.l 



HOST 

PROCESSOR 
UNIT 



-101 



GEOMETRIC 
COMPUTATION \~\02 
UNIT 



103 



RENDERING 
CONTROLLER 



DISPLAY 



105 



104 



SOUND 
GENERATOR 





SOUND 




MEMORY 



SOUND SYNTHESIZER 



SPEAKER 



106 



CD 
CD 

to 

CO 
O) 

o 

8 



> 



2/5 

HG.3A 



FIG.3B 



DRAWING 
COMMAND 



NORMAL DATA 1 
VERTEX DATA 1 



NORMAL DATA 2 
VERTEX DATA 2 



SOUND SOURCE 



SOUND SOURCE INDEX 

SOUND SOURCE POSITION DATA 



FIG.4 



HOST 

PROCESSOR 
UNIT 



-101 



i 



107 



GEOMETRIC 
COMPUTATION 
UNIT 



1 



108 



105 



GEOMETRIC 
COMPUTATION 
UNIT 



SOUND 
GENERATOR 



RENDERING 
CONTROLLER 



103 



A 



104 



SOUND 
MEMORY 



SOUND SYNTHESIZER 



DISPLAY 



SPEAKER 



T 

106 



1/5 

HG.l 



OR 



r-101 



gjftJPinATION 



103 r 



I 



-102 



RENDERING 
CONTROLLER 



DISPLAY 



105 



SOUND 
GENERATOR 



104 
-1 



SOUND 
MEMORY 



SOUND SYNTHESIZER 



SPEAKER 



106 



FIG.2 



Q LIGHT SOURCE 



VEHICLE X OBSERVER 
(SOUND SOURCE) * 0F LISTENING) 

SOUND 



13=3- 



A/WW- 



3/5 



FIG.5 



HOST 

PROCESSOR 
UNTT 



-HI 




DISPLAY 



108 



105 



GEOMETRIC 

COMPUTATION 

UNIT 



SOUND 
GENERATOR 



104 



SOUND 
MEMORY 



SOUND SYNTHESIZER 



SPEAKER 



V 
106 



FIG.6 



103 



HOST 

PRESSOR |~ 101 



rrwc 

COMPUTATION 
UNIT 



102 



RENDERING 
CONTROLLER 



105 

A. 



SOUND 
GENERATOR 



DISPLAY 



SOUND SYNTHESIZER 



SPEAKER 



"T" 

106 



4/5 

HG.7 



DIFFUSE 
SOUND 




AMBIENT 
SOUND 



SPOT SOUND 




DIRECTION OF 
SOUND EMISSION 



HG.8 



( START ) 



GENERATE 
AMBIENT SOUND 






GENERATE 
DIFFUSE SOUND 




t 


GENERATE SPOT 
SOUND 


* 


t 



~ST1 



~ ST2 



~ST3 



SYNTHESIZE 
AMBIENT SOUND, 
DIFFUSE SOUND 
AND SPOT SOUND 



~ ST4 



5/5 



FIG.9 

(PRIOR ART) 




GRAPHIC 

PROCESSING 

UNIT 



SOUND 
SOURCE 
PROCESSING 
UNIT 

r 

4 



5 



DBPLAY 
DEVICE 



SOUND 

EMITTING 

DEVICE 



T 
6 



1 2360681 

THREE-DIMENSIONAL SOUND REPRODUCTION 

SYSTEM 



The present invention generally relates 
to a three-dimensional sound reproduction system 
and, more particularly, to a three-dimensional 
sound reproduction system for reproducing three- 
dimensional graphics and sound in a three- 
dimensional space . 

In the accompanying drawings Fig. 9 shows a 
construction of a three- 
dimensional sound reproduction system according to 
the related art disclosed in Japanese Laid-Open 
Patent Application No. 8-149600. The system shown 
in Fig. 9 comprises a host system 1 comprising a 
personal computer, a game machine or the like, for 
executing a game program or a simulation; an object 
table 2 storing , for each of polygons constituting 
objects in three-dimensional computer graphics, 
three-dimensional coordinate values of vertices in 
the view coordinate system and displayed colors 
thereof. The object table 2 also stores sound 
information including tone, pitch and volume of a 
sound emitted from the vertices. The coordinate 
values of the vertices are modified on a real-time 
basis by the host system 1 in association with a 
change in a view point, movement of an object or 
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the like. 

The data stored In the object table 2 is 
supplied to a graphic processing unit 3 and a sound 
source processing unit 4. The graphic processing 
5 unit 3 determines the coordinates of the vertices 
in the screen coordinate system, based on the 
coordinates, stored in the object table 2, of the 
vertices of the polygons in the view coordinate 
system. The graphic processing unit 3 also executes 

10 hidden -line elimination and hidden- surf ace 

elimination, executes a three-dimensional computer 
graphic processes such as shading and texture 
mapping, based on the displayed colors, thus 
creating three-dimensional images on the screen of 

15 a display device 5. 

The sound source processing unit 4 
determines a position of a virtual sound source and 
a position of a polygon relative to the sound 
source, based on the coordinate values of the 

20 vertexes of the polygon stored in the object table 
2. The sound source processing unit 4 then 
generates sound via a sound emitting device 6, so 
that the sound is emitted from the position of the 
sound source thus identified toward the polygon, 

25 the sound being specified by associated sound 
Information. 

With the above -described construction of 
the related-art three-dimensional sound 
reproduction system, fine control of three- 

30 dimensional sound reproduction based on the 
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relationship between the sound source and a point 
of listening is difficult to achieve. A heavy load 
Is imposed on a host system such as a CPU running 
an application such as a game, thus reducing its 
5 performance, because the CPU is responsible for 
real-time computation of Information of the sound 
source and the point of listening, real- time 
converting of coordinate vaiues in three- 
dimensional graphics and real-time computation of 
10 color information. Thus, high-quality three- 
dimensional graphics and three-dimensional sound 
reproduction is hard to implement. 



15 Accordingly, an object of the present 

invention is to provide a three-dimensional sound 
reproduction system in which the aforementioned 
drawback is eliminated. 

Another and more specific object is to 

20 provide a three-dimensional sound reproduction 

system implementing high-quality three-dimensional 
graphics and three-dimensional sound reproduction 
on a real-time basis. 

The aforementioned objects can be 

2 5 achieved by a three-dimensional sound reproduction 
system for reproducing sound in a three-dimensional 
space in association with three-dimensional 
graphics, comprising: a host processor unit for 
storing graphic scene data and sound scene data 

30 constituting a three-dimensional scene, for 
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transferring drawing commands for drawing graphics 
and sound commands for reproducing sound and for 
controlling the entirety of the three-dimensional 
sound reproduction system; a geometric computation 
5 unit for performing geometric computation including 
conversion of coordinate values in three- 
dimensional graphics and computation of color 
information, based on the drawing command and the 
graphic scene data transferred from the host 

10 processor unit, so as to produce two-dimensional 

coordinate data and color data, and for identifying 
by computation geometric relationship between a 
sound source and a point of listening, based on the 
sound command and the sound scene data transferred 

15 from the host processor unit; a rendering 

controller for generating pixel data for display, 
based on the two-dimensional coordinate data and 
color data produced by the geometric computation 
unit; a sound memory for storing sound data of 

20 various sound sources; and a sound generator for 
reading out, from the sound memory, sound data 
corresponding to the sound command transferred from 
the host processor unit so as to generate sound in 
a three-dimensional space by processing the sound 

25 data thus read out. based on the geometric 

relationship between the sound source and the point 
of listening computed by the geometric computation 
unit* 

The geometric computation system may 
30 Include a first geometric computation unit for 



performing geometric computation including 
conversion of coordinate values in three- 
dimensional graphics and computation of color 
information, based on the drawing command and the 
graphic scene data transferred from the host 
processor unit, so as to produce two-dimensional 
coordinate data and color data; and a second 
geometric computation unit for identifying by 
computation geometric relationship between the 
sound source and the point of listening, based on 
the drawing command and the graphic scene data 
transferred from the host processor unit. 

The aforementioned objects can also be 
achieved by a three-dimensional sound reproduction 
system for reproducing sound in a three-dimensional 
space in association with three-dimensional 
graphics, comprising: a host processor unit for 
storing graphic scene data and sound scene data 
constituting a three-dimensional scene, for 
performing geometric computation including 
conversion of coordinate values in three- 
dimensional graphics and computation of color 
information, based on a drawing command for drawing 
graphics and graphic scene data, so as to produce 
two-dimensional coordinate data and color data, for 
transferring sound commands for reproducing sound, 
and for controlling the entirety of the three- 
dimensional sound, reproduction system; a rendering 
controller for generating pixel data for display, 
based on the two-dimensional coordinate data and 
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color data produced by the hot processor unit; a 
geometric computation unit for identifying by 
computation geometric relationship between a sound 
source and a point of listening, based on the sound 
5 commend and the sound scene data transferred from 
the host processor unit: a sound memory for storing 
sound data of various sound sources; and a sound 
generator for reading out, from the sound memory, 
sound data corresponding to the sound command 

10 transferred from the host processor unit so as to 
generate sound in a three-dimensional space by 
processing the sound data thus read out, based on 
the geometric relationship between the sound source 
and the point of listening computed by the 

15 geometric computation unit. 

The sound memory may store ambient sound 
data for ambient sound in the three-dimensional 
space, diffuse sound data for non-directional 
diffuse sound transmitted directly from the sound 

20 source, and spot sound data for directional spot 
sound transmitted directly from the sound source; 
and the sound generator may generate the ambient 
sound in the three-dimensional space by reading out, 
from the sound memory, the ambient sound data for 

25 the ambient sound specified by the sound command 
transmitted from the host processor unit; generate 
the diffuse sound in the three-dimensional space by 
reading out, from the sound memory, the diffuse 
sound data for the diffuse sound specified by the 

30 sound command from the host processor unit and by 
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processing the diffuse sound data thus read out, 
based on the geometric relationship between the 
sound source and the point of listening computed by 
the geometric computation unit; generate the spot 
5 sound in the three-dimensional space by reading out, 
from the sound memory, the sound data for the spot 
sound specified by the sound command transferred 
from the host processor unit and by processing the 
sound data for the spot sound thus read out, based 

10 on the geometric relationship between the sound 
source and the point of listening computed by the 
geometric computation unit; and synthesize the 
ambient sound, the diffuse sound and the spot sound 
thus generated. 

15 The aforementioned objects can also be 

achieved by a three-dimensional sound reproduction 
system for reproducing sound in a three-dimensional 
space in association with three-dimensional 
graphics, comprising: a host prooessbr unit for 

20 storing graphic scene data and sound scene data 
constituting a three-dimensional scene, for 
transferring drawing commands for drawing graphics 
and sound commands for reproducing sound and for 
controlling the entirety of the three-dimensional 

25 sound reproduction system, the sound scene data 
including sound data for various sound sources; a 
geometric computation unit for performing geometric 
computation including conversion of coordinate 
values in three-dimensional graphics and 

30 computation of color information, based on the 
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drawing command and the graphic scene data 
transferred from the host processor unit, so as to 
produce two-dimensional coordinate data and oolor 
data, and for identifying by computation geometric 
5 relationship between a sound source and a point of 
listening, based on the sound command and the sound 
scene data transferred from the host processor 
unit; a rendering controller for generating pixel 
data for display, based on the two-dimensional 

10 coordinate data and oolor data produced by the 
geometric computation unit; a sound memory for 
storing sound data of various sound sources; and a 
sound generator for reading out, from the host 
processor unit, the sound data corresponding to the 

15 sound command transferred from the host processor 

unit so as to generate sound in a three-dimensional 
space by processing the sound data thus read out, 
based on the geometric relationship between the 
sound source and the point of listening computed by 

20 the geometric computation unit. 

The host processor unit may store 
ambient sound data for ambient sound in the three- 
dimensional space, diffuse sound data for non- 
directional diffuse sound transmitted directly from 

25 the sound source, and spot sound data for 

directional spot sound transmitted directly from 
the sound source; and the sound generator may 
generate the ambient sound in the three-dimensional 
space by reading out, from the host processor unit, 

30 the ambient sound data for the ambient sound 
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specified by the sound command transmitted from the 
host processor unit; generate the diffuse sound in 
the three-dimensional space by reading out, from 
the host processor unit, the diffuse sound data for 
5 the diffuse sound specified by the sound command 
from the host processor unit and by processing the 
diffuse sound data thus read out, based on the 
geometric relationship between the sound source and 
the point of listening computed by the geometric 
10 computation unit? generate the spot sound in the 
three-dimensional space by reading out. .from the 
host processor unit, the sound data for the spot 
sound specified by the sound command transferred 
from the host processor unit and by processing the 
15 sound data for the spot sound thus read out, based 
on the geometric relationship between the sound 
source and the point of listening computed by the 
geometric computation unit; and synthesize the 
ambient sound, the diffuse sound and the spot sound 
20 thus generated. 

The host processor unit may store sound 
data such as that of background music not related 
to the three-dimensional space, and the three- 
dimensional sound reproduction system may further 
25 comprise a sound synthesizer for synthesizing the 
sound in the three-dimensional space generated by 
the sound generator with the sound data from the 
host processor unit not related to the three- 
dimensional space. 



30 
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The present invention will be further described by 
way of non-limitative example with reference to the 
accompanying drawings, in which: 



Pig. 1 shows the construction of a 
three-dimensional sound reproduction system 
according to a first embodiment of the present 
invention; 

Pig. 2 illustrates a relationship 
between a moving vehicle and an observer according 
to the first embodiment; 

Figs. 3A and 38 show commands 
transferred by a host processor unit according to 
the first embodiment; 

Pig. 4 shows the construction of a 
three-dimensional sound reproduction system 
according to a second embodiment of the present 
invention; 

Pig. 5 shows the construction of a 
three-dimensional sound reproduction system 
according to a third embodiment of the present 
invention; 

pig. 6 shows the construction of a 
three-dimensional sound reproduction system 
according to a fourth embodiment of the present 
invention; 

Fig. 7 illustrates types of sound 
simulated according to the fifth embodiment of the 
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present Invention; 

Fig, 8 is a flowchart showing a process 
executed by a sound generator according to the 
fifth embodiment; and 

Fig. 9 shows a construction of a three- 
dimensional sound reproduction system according to 
the related art. 

First embodiment 

Fig. 1 shows a construction of a three- 
dimensional sound reproduction system according to 
a first embodiment. Referring to Fig. 1, the three- 
dimensional sound reproduction system comprises a 
host processor unit 101 for storing graphic scene 
data and sound scene constructing a three- 
dimensional scene. The host processor unit 101, 
controlling the entire system, transfers drawing 
commands for drawing graphics and sound commands 
for reproducing sound. The host processor unit 101 
also stores sound data not related to the three- 
dimensional space such as background music and beep 
sound (alert sound). 

Referring also to Fig. 1, the three- 
dimensional sound reproduction system also comprise 
a geometric computation unit 102 for performing 
geometric computation Including conversion of 
coordinate values in the three-dimensional graphics 
and computation of color information, based on the 
drawing command and the graphic scene data 
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transferred from the host processor unit 101, so as 
to produce two-dimensional objeot data comprising 
the coordinate data and color data. The geometric 
computation unit 102 also identifies by computation 
5 geometric relationship such as a distance between 
the sound source and the point of listening, and a 
magnitude of movement. 

Referring also to Fig. 1, the three- 
dimensional Bound reproduction system also comprise 
10 a rendering controller 103 for generating pixel 
data for display, based on the two-dimensional 
object data comprising the coordinate data and the 
color data computed by the geometric computation 
unit 102. 

15 Referring also to Fig. 1, the three- 

dimensional sound reproduction system also 
comprises a sound memory 104 for storing sound data 
of various sound sources, a sound generator 105 for 
reading out, from the sound memory 10 4, sound data 

20 corresponding to a sound command transferred from 
the host processor unit so as to generate sound in 
a three-dimensional space by processing the sound 
data thus read out, based on the distance and the 
magnitude of movement computed by the geometric 

25 computation unit 102. The three-dimensional sound 
reproduction system also comprises a sound 
synthesizer 106 for synthesizing the three- 
dimensional sound generated by the sound generator 
105 and the sound not related to the three- 

30 dimensional space read out from the host processor 
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101. 

A description will now be given of the 
operation according to the first embodiment. 

The mode by which the sound is heard 
5 depending on the movement of an object, a point of 
viewing or a point of listening is controlled by 
mapping a sound source to a three-dimensional 
object in three-dimensional graphics. Fig. 2 
illustrates a relationship between a moving vehicle 

10 and an observer according to the first embodiment. 
It is assumed that a vehicle (sound source) 
emitting a sound, an observer (point of listening) 
and light source are provided in the three- 
dimensional graphics, as shown in Fig. 2. Assuming 

15 that the observer is viewing the vehicle, matrix 

conversion of the three-dimensional graphics into a 
point of viewing coordinate system having the eyes 
of the observer at the origin is conducted. 
Computation of color information related to the 

20 vehicle is performed at the positions of the light 
source, the vehicle and the eyes, respectively. 

The three-dimensional data is converted 
into two-dimensional data so that the two- 
dimensional image is filled with color. The three - 

25 dimensional graphic process is applied to the sound 
source and the observer. Matrix conversion into a 
point of viewing coordinate system having the 
observer at the origin is conducted so that the 
relative and dynamic geometric relationship between 

30 the sound source and the observer is identified. 
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The sound emitted from the vehicle ie controlled 
three-dimensionally, based on the information thus 
collected. 

The host processor 101 stores the 
5 graphic scene data such as vertex data, normal data 
and light source data of an object, such as a 
vehicle, constructing a three-dimensional scene. 
Fig. 3 Figs. 3A and 3B show commands transferred by 
the host processor 101 unit according to the first 
10 embodiment. Fig. 3A shows a drawing command for 
graphics and Fig. 3.B shows a sound command. 

In order to draw a scene and reproduce 
sound therefor, the host processor 101 transfers a 
plurality of drawing commands and sound commands, 
15 graphic scene data corresponding to the drawing 
command and sound scene data corresponding to the 

s 

sound command, to the geometric computation unit 
102. In transferring the commands, it is not 
necessary to differentiate between the drawing 

20 command and the sound command. The commands may be 
transferred in bulk using the dlreot memory access 
(DMA) transfer. Related-art software for allowing 
the host processor unit 101 of the three- 
dimensional graphic system to handle the data may 

25 be used to transfer the commands. 

The geometric computation unit 102 
converts three-dimension data into two-dimensional 
data by model view conversion defined in a graphic 
library such as OpenGL, clipping, color computation 

30 and device conversion, so as to produce object data 
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comprising two-dimensional coordinate data and 
color data. The rendering controller 103 fills he 
two-dimensional image with color, based on the 
object data comprising the coordinate data and the 
5 color data produced by the geometric computation 
unit 102, so as to generate pixel data displayed on 
the display device. 

Responsive to the sound command, the 
geometric computation unit 102 identifies by 

10 computation the geometric relationship such as the 
distance between the sound source and the point of 
listening, and the magnitude of movement. Using the 
model view conversion as used in the three- 
dimensional graphics library, conversion into the 

15 point of viewing (point of listening) coordinate 
system having the eyes (ears) of the observer at 
the origin so that the relative position of the 
point of listening and the sound source is 
Identified by computation. Information relating to 

20 the movement of the sound source or the point of 
listening should be determined in the world 
coordinate system before the conversion into the 
model view. 

The sound generator 105 reads out, from 

25 the sound memory 104, the sound data corresponding 
to the sound source specified by the sound command 
and transferred from the host processor unit 101. 
The sound generator 105 processes the sound data 
thus read out, based on the geometric Information 

30 relating to the sound source and the point of 
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listening and computed by the geometric computation 
unit 120. so as to generate sound in the three- 
dimensional apaoe. Processes performed by the sound 
generator 105 Include control of a sound level 
5 depending on the distance between the sound source , 
and the point of listening, applying of the Doppler 
effect baeed on the information relating to the 
movement of the sound source and the point of 
listening, control of a sound level depending on 

10 the direction in vhioh the sound is emitted. 

The sound synthesizer 106 synthesizes 
the sound in the three-dimensional space generated 
by the sound generator 105 and the sound not 
related to the three-dimensional apace such as 

15 background music and a beep read out from the host 
processor unit 101. The synthesized sound is output 
from the Bound synthesizer 106. 

According to the first embodiment, the 
sound not related to the three -dimensional space 

20 suoh as background mnslo and a beep are stored by 
the host processor unit 101. The sound synthesizer 
106 synthesizes the sound in the three-dimensional 
space generated by the sound generator 105 and the 
sound not related to the three-dimensional space * 

25 read out from the host processor unit 101. The 
sound synthesizer 106 may not be provided in the 
three-dimensional sound reproduction system. By 
providing the sound synthesizer 106, the sound in 
the three-dimensional space may contain desired 

30 sound not related to the three-dimensional space. 
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As described above, according to the 
first embodiment, the sound source data for three- 
dimensional sound synthesis is processed as an 
object in the three-dimensional graphics. The host 
processor 101 transfers the drawing command, the 
sound command, the graphic scene data, the sound 
scene data to the geometric computation unit 102. 
The host processor unit 101 is relieved of a load 
of extra processes. The geometric computation unit 
102 generates the three-dimensional graphics. The 
sound generator 105 generates the sound in the 
three-dimensional space, based on the result of 
computation by the geometric computation unit 102. 
With this, high-quality three-dimensional graphics 
and high-quality three-dimensional sound are 
reproduced on a real-time basis. 

Second embodiment 

Pig. 4 shows the construction of a 
three-dimensional sound reproduction system 
according to a second embodiment of the present 
invention. Referring to Pig. 4. the three- 
dimensional sound reproduction system comprises a 
geometric computation unit 107 for performing 
geometric computation including conversion of 
coordinate values in the three-dimensional graphics 
and computation of color Information, based on the 
drawing command and the graphic scene data 
transferred from the host processor unit 101, so as 
to produce two-dimensional object data comprising 
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the coordinate data and oolor data. The three- 
dimensional sound reproduction system aooording to 
the second embodiment also comprises a geometric 
computation unit 108 for identifying by computation 
5 geometric relationship such as a distance between 
the sound souroe and the point of listening, and a 
magnitude of movement , based on the sound command 
and the sound scene data transferred from the host 
processor unit 101. The other aspects of the 
10 construction remains the same as the corresponding 
aspects aooording to the first embodiment shown in 
Pig. 1. 

While the three-dimensional graphic 
process and the three-dimensional sound process are . 

15 performed by the geometric computation unit 102 
according to the first embodiment, the second 
embodiment Is configured such that the three- 
dimensional graphic process and the three- 
dimensional sound process are separated. 

20 A description will now be given of the 

operation according to the second embodiment. 

The host processor unit 101 manages the 
drawing command and the sound command shown in Figs . 
3A and 3B Independently. The host processor unit 

25 101 transfers the drawing command and the graphic 
scene data to the geometric computation unit 107 
and transfers the sound command and the sound scene 
data to the geometric computation unit 108. The 
geometric computation unit 107 performs the three- 

30 dimensional graphic process and the geometric 
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computation unit 108 performs the three-dimensional 
sound process. The other processes according to the 
second embodiment are the same as the corresponding 
processes according to the first embodiment. 

As has been described, according to the 
second embodiment, it is possible to process the 
sound source data for three-dimensional sound 
generation is processed as an object in the three- 
dimensional graphics. The host processor 101 
transfers the drawing command and the graphic scene 
data to the geometric computation unit 107 , 
transfers the sound command and the sound scene 
data to the geometric computation unit 108. The 
host processor unit 101 is relieved of a load of 
extra processes. The geometric computation unit 107 
generates the three-dimensional graphics. The sound 
generator 105 generates the sound in the three- 
dimensional space, based on the result of 
computation by the geometric computation unit 108. 
With this, high-quality three-dimensional graphics 
and high-quality three-dimensional sound are 
reproduced on a real-time basis. 

Third embodiment 

Pig. 5 shows the construction of a 
three-dimensional sound reproduction system 
according to a third embodiment of the present 
invention. Referring to Fig. 5, the three- 
dimensional sound reproduction system comprises a 
host processor unit 111 for controlling the entire 
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system by storing graphic scene data and sound 
scene data constituting a three-dimensional scene, 
performing geometric computation Including 
conversion of coordinate values In the three- 
5 dimensional graphics and computation of color 

Information, based on the drawing command and the 
graphic scene data, so as to produce two- 
dimensional object data comprising the coordinate 
data and color data, and transferring the sound 

10 command. The host processor unit 111 also stores 
sound data not related to the three-dimensional 
space such as background music and beep sound 
(alert sound). 

Referring to Fig. 5, the other aspects 

15 of the construction according to the second 
embodiment are the same as the corresponding 
aspects of the second embodiment shown in Fig. 4 
except that the geometric computation unit 107 is 
eliminated. The host processor unit 111 shown in 

20 Fig. 5 includes the function of the geometric 
computation unit 107. 

The geometric computation unit 102 of 
the first embodiment performs the three-dimensional 
graphic process and the three-dimensional sound 

25 process. According to the third embodiment, the 
three-dimensional graphic process and the three- 
dimensional sound process are separated from each 
other. The three-dimensional graphic process is 
performed by the host processor unit 111 and the 

30 three-dimensional sound process is performed by the 
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geometric computation unit 108. 

A description will now be given of the 
operation according to the third embodiment. 

The host processor unit 111 manages the 
drawing command and the sound command shown in Figs. 
3A and 3B independently. The host processor unit 
111 transfers the sound command and the sound scene 
data to the geometric computation unit 108 and 
performs geometric computation including conversion 
of coordinate values in the three-dimensional 
graphics and computation of color information, 
based on the drawing command and the graphic scene 
data, so as to produce two-dimensional object data 
comprising the coordinate data and color data. 

The rendering controller 103 fills he 
two- dimensional image with color, based on the 
object data comprising the coordinate data and the 
color data produced by the host processor unit 111, 
so as to generate pixel data displayed on the 
display device. The other aspects of the process 
according to the third embodiment are the same as 
the corresponding aspects according to the second 
embodiment . 

As described above, according to the 
third embodiment, the sound source data for three- 
dimensional sound synthesis is processed as an 
object in the three-dimensional graphics. The host 
processor unit 111 generates three-dimensional 
graphics, transfers the sound data and the sound 
scene data to the geometric computation unit 108. 
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The Bound generator 105 generates the sound In the 
three-dimensional space, based on the result of 
computation by the geometric computation unit 108 • 
With this, high-quality three-dimensional graphics 
5 and high-quality three-dimensional sound are 
reproduced on a real-time basis. 

Fourth embodiment 

Fig. 6 shows the construction of a 

10 three-dimensional sound reproduction system 

according to a fourth embodiment of the present 
invention. The construction according to the sixth 
embodiment differs from that of the first 
embodiment shown in Pig. 1 in that the sound memory 

15 104 is eliminated. The first embodiment is 

configured such that the sound memory 104 stores 
sound data for various sound sources. In the fourth 
embodiment , however, the sound data for various 
sound sources is stored as part of scene data in 

20 the host processor unit 111. With this, an 

inexpensive three-dimensional sound system is 
constructed. 

A description will now be given of the 
operation according to the fourth embodiment. 

25 The sound generator 105 reads out sound 

data corresponding to a sound source specified by 
the sound command from the host processor unit 101 
and processes the sound data, based on the 
geometric Information relative to the sound source 

30 and the point of listening input from the geometric 
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computation unit 102. The other aspects of the 
fourth embodiment are the same as the corresponding 
aspects according to the first embodiment. 

As described above, according to the 
fourth embodiment, the sound data for three- 
dimensional sound synthesis is processed as an 
object of the three-dimensional graphics. The host 
processor 101 transfers the drawing command, the 
sound command, the graphic scene data, the sound 
scene data to the geometric computation unit 102. 
The host processor unit 101 is relieved of a load 
of extra processes. The geometric computation unit 
102 generates the three-dimensional graphics. The 
sound generator 105 generates the sound in the 
three-dimensional space, based on the result of 
computation by the geometric computation unit 102. 
The sound data for the sound sources is stored in 
the host processor unit 101. With this, high- 
quality three-dimensional graphics and high-quality 
three-dimensional sound are reproduced on a real- 
time basis using an Inexpensive construction. 

Fifth embodiment 

The construction of the three- 
dimensional sound reproduction system according to 
the fifth embodiment is the same as the 
construction according to the first embodiment 
shown in Pig. l. The fifth embodiment is configured 
such that the geometric computation unit for three- 
dimensional graphics is facilitated by simulating 
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sound as well as light, thereby reproducing 
realistic sound. 

Fig. 7 illustrates typeB of sound 
simulated according to the fifth embodiment of the 
present invention, as shown in Pig. 7, a 
classification of sound into ambient sound and spot 
sound is given. The ambient sound does not have any 
explicit sound source but is ubiquitous throughout 
the entire three-dimensional space. Diffuse sound 
is non-directional sound direct from a sound source. 
Spot sound is emitted from a directional sound 
source. The total profile of sound is build by 
synthesizing these types of sound. The sound data 
for ambient sound, the sound data for diffuse sound 
and the sound data for spot sound are stored in the 
sound memory 104. 

A description will now be given of the 
operation according to the fifth embodiment. 

Fig. 8 is a flowchart showing a process 
executed by a sound generator according to the 
fifth embodiment. In step BT1, the sound generator 
105 reads out sound data for ambient sound stored 
in the sound memory 104, based on the sound command 
transferred from the host processor unit 101. The 
sound generator 105 also generates, in the three- 
dimensional space, ambient sound (noise-like sound) 
analogous to ambient light of computer graphics. 

In step ST2, the sound generator 105 
reads out from the sound memory 104 the sound data 
for diffuse sound corresponding to the sound source 
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specified by the sound command transferred from the 
host processor unit 101 . The sound generator 105 
processes the sound data for diffuse sound thus 
read out, based on the geometric information 
5 relating to the sound source and the point of 

listening and computed by the geometric computation 
unit 102. Thereby, the sound generator 105 
generates, in the three-dimensional space, the 
diffuse sound analogous to diffuse light of 

10 computer graphics. The diffuse sound is directly 
transmitted from the sound source in a non- 
directional manner. 

In step ST3, the sound generator 105 
reads out from the sound memory 10 4 the sound data 

15 for spot sound corresponding to the sound source 

specified by the sound command transferred from the 
host processor unit 101. The sound generator 105 
processes the sound data for spot sound thus read 
out, based on the geometric information relating to 

20 the sound source and the point of listening and 
computed by the geometric computation unit 102. 
Thereby, the sound generator 105 generates, in the 
three-dimensional space, the spot sound analogous 
to spot light of oomputer graphics. The spot sound 

25 is directly transmitted from the sound source in a 
directional manner. 

In step ST4, the ambient sound in the 
three-dimensional space generated in step ST1, the 
diffuse sound in the three-dimensional space 

30 generated in step ST2, the spot sound in the three- 
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dimensional space generated In step ST3 are 
synthesized so as to produce the entirety of the 
sound In the three-dimensional space. By processing 
the sound In the same manner as light Is simulated, 
5 the sound In a three-dimensional space Is easily 
simulated. The other aspects of the process are the 
same as the corresponding aspects according to the 
first embodiment. 

The construction of the three- 

10 dimensional sound reproduction system according to 
the fifth embodiment could be the same as the 
construction according to the first embodiment 
shown in Fig. 1, the construction according to the 
second embodiment shown in Pig. 4 or the 

15 construction according to the third embodiment 

shown in Fig. 5. The construction according to the 
fifth embodiment could also be the same as the 
construction according to the fourth embodiment 
shown in Fig. 6, as long as the host processor 101 

20 of Fig. 6 8 tores the sound data for ambient sound, 
the sound data for diffuse sound and the sound data 
for spot sound. 

As described above, according to the 
fifth embodiment, the sound data for generating the 

25 three-dimensional sound is processed as an object 
in the three-dimensional graphics. The host 
processor unit 101 transfers the drawing command, 
the sound command, the graphic scene data and the 
sound scene data to the geometric computation unit 

30 102. The host processor unit 101 is not required to 
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perform extra processes since the geometric 
computation unit 102 generates the three- 
dimensional graphics and the sound generator 10 5 
simulates the sound In the three-dimensional sound 
5 according to the type of sound, based on the result 
of computation by the geometric computation unit 
102. With this, real-time high-quality three- 
dimensional sound reproduction according to the 
type of sound and high-quality three-dimensional 
10 graphic reproduction Is implemented. 

The present invention is not limited to 
the above-described embodiments, and variations and 
modifications may be made without departing from 
the scope of the present Invention. 
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CLAIMS 



1. A three-dimensional sound 
reproduction system for reproducing sound in a 
three-dimensional space in association with throe- 
dimensional graphics, comprising: 

a host processor unit for storing 
graphic scene data and sound scene data 
constituting a three-dimensional scene, for 
transferring drawing commands .f. or drawing graphics 
and sound commands for reproducing sound and for 
controlling the entirety of the three-dimensional 
sound reproduction system; 

a geometric computation unit for 
performing geometric computation Including 
conversion of coordinate values in three- 
dimensional graphics and computation of color 
information, based on the drawing command and the 
graphic scene data transferred from said host 
processor unit, so as to produce two-dimensional 
coordinate data and color data, and for identifying 
by computation geometric relationship between a 
sound source and a point of listening, based on the 
sound command and the sound scene data transferred 
from said host processor unit; 

a rendering controller for generating 
pixel data for display, based on the two- 
dimensional coordinate data and color data produced 
by said geometric computation unit; 

a sound memory for storing sound data of 
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various sound sources; and 

a sound generator for reading out, from 
said sound memory, sound data corresponding to the 
sound command transferred from said host processor 
unit so as to generate sound in a three-dimensional 
space by processing the sound data thus read out. 
based on the geometric relationship between the 
sound source and the point of listening computed by 
said geometric computation unit. 

2. The three-dimensional sound 
reproduction system according to claim 1. wherein 
said geometric computation system includes a first 
geometric computation unit for performing geometric 
computation including conversion of coordinate 
values in three-dimensional graphics and 
computation of color information, based on the 
drawing command and the graphic scene data 
transferred from said host processor unit, so as to 
produce two-dimensional coordinate data and color 
data; and a second geometric computation unit for 
identifying by computation geometric relationship 
between the sound source and the point of listening, 
based on the drawing command and the graphic scene 
data transferred from said host processor unit. 

3. A three-dimensional sound 
reproduction system for reproducing sound in a 
three-dimensional space in association with three- 
dimensional graphics, comprising: 
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a host processor unit for storing 
graphic scene data and sound scene data 
constituting a three-dimensional scene, for 
performing geometric computation including 
conversion of coordinate values in three- 
dimensional graphics and computation of color 
information, based on a drawing command for drawing 
graphics and graphic scene data, so as to produce 
two-dimensional coordinate data and color data, for 
transferring sound commands for reproducing sound, 
and for controlling the entirety of the three- 
dimensional sound reproduction system; 

a rendering controller for generating 
pixel data for display, based on the two- 
dimensional coordinate data and color data produced 
by said hot processor unit; 

a geometric computation unit for 
identifying by computation geometric relationship 
between a sound source and a point of listening, 
based on the sound command and the sound scene data 
transferred from said host processor unit; 

a sound memory for storing sound data of 
various sound sources; and 

a sound generator for reading out, from 
said sound memory, sound data corresponding to the 
sound command transferred from said host processor 
unit so as to generate sound in a three-dimensional 
space by processing the sound data thus read out, 
based on the geometric relationship between the 
sound source and the point of listening computed by 
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said geometric computation unit. 

4, The three-dimensional sound 
reproduction system according to claim 1., wherein 
5 said sound memory stores ambient sound data for 
ambient sound in the three-dimensional space, 
diffuse sound data for non-directional diffuse 
sound transmitted directly from the sound source, 
and spot sound data for directional spot sound 
10 transmitted directly from the. sound source; and 
wherein 

said sound generator generates the 
ambient sound in the three-dimensional space by 
reading out, from said sound memory, the ambient 

15 sound data for the ambient sound specified by the 
sound command transmitted from said host processor 
unit; generates the diffuse sound in the three- 
dimensional space by reading out, from said sound 
memory, the diffuse sound data for the diffuse 

20 sound specified by the sound command from said host 
processor unit and by processing the diffuse sound 
data thus read out, based on the geometric 
relationship between the sound source and the point 
of listening computed by said geometric computation 

25 unit; generates the spot sound in the three- 
dimensional space by reading out, from said sound 
memory, the sound data for the spot sound specified 
by the sound command transferred from said host 
processor unit and by processing the sound data for 

3 0 the spot sound thus read out, based on the 
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geometric relationship between the sound souroe and 
the point of listening computed by said geometric 
computation unit; and synthesizes the ambient sound, 
the diffuse sound and the spot sound thus generated. 

5 

5. The three-dimensional sound 
reproduction system according to claim 3, wherein 
said sound memory stores ambient sound data for 
ambient sound in the three-dimensional space, 
10 diffuse sound data for non-direptional diffuse 

sound transmitted directly from the sound source, 
and spot sound data for directional spot sound 
transmitted directly from the sound source; and 
wherein 

15 said sound generator generates the 

ambient sound in the three-dimensional space by 
reading out, from said sound memory, the ambient 
sound data for the ambient sound specified by the 
sound command transmitted from said host processor 

20 unit; generates the diffuse sound in the three- 
dimensional space by reading out, from said sound 
memory, the diffuse sound data for the diffuse 
sound specified by the sound command from said host 
processor unit and by processing the diffuse sound 

25 data thus read out, based on the geometric 

relationship between the sound souroe and the point 
of listening computed by said geometric computation 
unit; generates the spot sound in the three- 
dimensional space by reading out, from said sound 

30 memory, the sound data for the spot sound specified 
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by the sound command transferred from said host 
processor unit and by processing the sound data for 
the spot sound thus read out, based on the 
geometric relationship between the sound source and 
the point of listening computed by said geometric 
computation unit; and synthesizes the ambient sound, 
the diffuse sound and the spot sound thus generated. 

6. A three-dimensional sound 
reproduction system for reproduping sound. in a 
three-dimensional space in association with three- 
dimensional graphics, comprising: 

a host processor unit for storing 
graphic scene data and sound scene data 
constituting a three-dimensional scene, for 
transferring drawing commands for drawing graphics 
and sound commands for reproducing sound and for 
controlling the entirety of the three-dimensional 
sound reproduction system, the sound scene data 
including sound data for various sound sources; 

a geometric computation unit for 
performing geometric computation including 
conversion of coordinate values in three- 
dimensional graphics and computation of color 
information, based on the drawing command and the 
graphic scene data transferred from said host 
processor unit, so as to produce two-dimensional 
coordinate data and color data, and for identifying 
by computation geometric relationship between a 
sound source and a point of listening, based on the 
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sound command and the sound scene data transferred 
from said host processor unit; 

a rendering controller for generating 
pixel data for display, based cn the two- 
dimensional coordinate data and color data produced 
by said geometric computation unit; 

a sound memory for storing sound data of 
various sound sources; and 

a sound generator for reading out, from 
said host processor unit, the . spund data 
corresponding to the sound command transferred from 
said hcst processor unit so as tc generate sound in 
a /three-dimensional space by processing the sound 
data thus read out, based on the geometric 
relationship between the sound source and the point 
of listening computed by said geometric computation 
unit . 

7. The three-dimensional sound 
reproduction system according to claim 6, wherein 
said host processor unit stores ambient sound data 
for ambient sound in the three-dimensional space, 
diffuse sound data for non-directional diffuse 
sound transmitted directly frcm the sound source, 
and spot sound data for directional spot sound 
transmitted directly from the sound source; and 
wherein 

said sound generator generates the 
ambient sound in the three-dimensional space by 
reading out, from said host processor unit, the 
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ambient sound data for the ambient sound specified 
by the sound command transmitted from said host 
processor unit; generates the diffuse sound in the 
three-dimensional space by reading out, from said 
5 host processor unit, the diffuse sound data for the 
diffuse sound specified by the sound command from 
said host processor unit and by processing the 
diffuse sound data thus read out, based on the 
geometric relationship between the sound source and 

10 the point of listening computed, by said geometric 
computation unit; generates the spot sound in the 
three-dimensional space by reading out, from said 
host processor unit, the sound data for the spot 
sound specified by the sound command transferred 

15 from said host processor unit and by processing the 
sound data for the spot sound thus read out, based 
on the geometric relationship between the sound 
source and the point of listening computed by said 
geometric computation unit; and synthesizes the 

20 ambient sound, the diffuse sound and the spot sound 
thus generated. 

8. The three-dimensional sound 
reproduction system according to claim 1, wherein 

25 said host processor unit stores sound data such as 
that of background music not related to the three- 
dimensional space, said three-dimensional sound 
reproduction system further comprising a sound 
synthesizer for synthesizing the sound in the 

30 three-dimensional space generated by said sound 
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generator with the sound data from said host 
processor unit not related to the three-dimensional 
space. 

5 9. The three-dimensional sound 

reproduotion system according to claim 3, wherein 
said host processor unit stores sound data such as 
that of background music not related to the three- 
dimensional space, said three-dimensional sound 

10 reproduction system further comprising a sound 
synthesizer for synthesizing the sound in the 
three-dimensional space generated by said sound 
generator with the sound data from said host 
processor unit not related to the three-dimensional 

15 space. 

10. The three-dimensional sound 
reproduction system according to claim 6, wherein 
said host processor unit stores sound data such as 

20 that of background music not related to the three- 
dimensional spacer said three-dimensional sound 
reproduction system further comprising a sound 
synthesizer for synthesizing the sound in the 
three-dimensional space generated by said sound 

25 generator with the sound data from said host 

processor unit not related to the three-dimensional 
space. 

11. A three-dimensional sound 

30 reproduction system substantially as herein 
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described with reference to any of Figures 1 to 8 
of the accompanying drawings . 



Amendments to the claims have been filed as follows 

CLAIMS 

1. A three-dimensional sound 
reproduction system for reproducing sound in a 
three-dimensional space in association with three- 
dimensional graphics, comprising: 

a host processor unit for storing 
graphic scene data and sound scene data 
constituting a three-dimensional scene, for 
transferring drawing commands . fpr drawing graphics 
and sound commands for reproducing sound and for 
controlling the entirety of the three-dimensional 
sound reproduction system; 

a geometric computation unit for 
performing geometric computation including 
conversion of coordinate values in three- 
dimensional graphics and computation of color 
information, based on the drawing command and the 
graphic scene data transferred from said host 
processor unit, so as to produce two-dimensional 
coordinate data and color data, and for identifying 
by computation geometric relationship between a 
sound source and a point of listening, based on the 
sound command and the sound scene data transferred 
from said host processor unit; 

a rendering controller for generating 
pixel data for display, based on the two- 
dimensional coordinate data and color data produced 
by said geometric computation unit; 

a sound memory for storing sound data of 



various sound sources* and 

a sound generator for reading out, from 
said sound memory, sound data corresponding to the 
sound command transferred from said host processor 
unit so as to generate sound in a three-dimensional 
space by processing the sound data thus read out. 
based on the geometric relationship between the 
sound source and the point of listening computed by 
said geometric computation unit. 

2. The three-dimensional sound 
reproduction system according to claim l. wherein 
said geometric computation system includes a first 
geometric computation unit for performing geometric 
computation including conversion of coordinate 
values in three-dimensional graphics and 
computation of color information, based on the 
drawing command and the graphic scene data 
transferred from said host processor unit, so as to 
produce two-dimensional coordinate data and color 
data; and a second geometric computation unit for 
identifying by computation geometric relationship 
between the sound source and the point of listening, 
based on the drawing command and the graphic scene 
data transferred from said host processor unit. 

3. A three-dimensional sound 
reproduction system for reproducing sound in a 
three-dimensional space in association with three- 
dimensional graphics, comprising: 



geometric relationship between the sound source and 
the point of listening computed by said geometric 
computation unit; and synthesizes the ambient sound, * 
"the diffuse sound and th.e- spot* sound thus generated. 

5 

5. A three-dimensional sound 
reproduction system for reproducing sound. in a 
three-dimensional space in association with three- 
■ dimensional graphics, comprising: 

10 a host processor unit for storing 

graphic scene data and- sound scene data 
constituting a three-dimensional scene, for 
transferring drawing commands for drawing graphics 
and sound commands for reproducing sound and for 

15 controlling the entirety of the three-dimensional 
sound reproduction system, the sound scene data 
including sound data for various sound sources; 

a geometric computation unit for 
performing geometric computation Including 

20 conversion of coordinate values in three- 
dimensional- graphics and computation of color 
information, based on the drawing command and the 
graphic scene data transferred from said host 
processor unit, so as to produce two-dimensional 

25 coordinate . data and color data, and for identifying 
by computation geometric relationship between a 
sound source and a point of listening, based on the 
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sound command and '.the sound scene data transferred 
from said host prpcessor unit; 

a rendering controller for generating 
► pixej. data for display, based on the two- 
dimensional coordinate data and color data produced 
by said geometric .computation unit; and 

a sound generator -for reading out, from 
said host processor unit, the. sound data 
corresponding to the spund. command transferred from 
said host processor unit so as to generate sound in 
a three-dimensional space by processing the sound 
data thus read out, based on the geometric 
relationship between the sound source and the point 
of listening computed by said geometric computation 
unit . 

i 

6. The three-dimensional sound 
reproduction system according to claim 5, wherein 
said host processor unit stores ambient sound data 
for ambient sound in the three-dimensional space, 
diffuse sound data for non-directional diffuse 
sound transmitted directly from the sound source, 
and spot sound data for directional spot sound 
transmitted directly from the sound source; and 
wherein 

said sound generator generates the 
ambient sound in the three-dimensional space by 
reading out, from said host processor unit, the 
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